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— — *  We  consider  a  simple  model  of  overhead  in  batch  computer  systems  and  message  switching 
systems:  an  exponential,  single-server  queuing  system  with  finite  storage  capacity,  constant  arrival 
rate,  and  queue-length-dependent  service  time.  We  consider  cases  in  which  the  expected  service  time 
time  consists  of  a  constant  plus  a  term  that  grows  linearly  or  logarithmically  with  the  queue  length. 

We  show  that  the  performance  of  this  system  —  as  characterized  by  the  expected  number  of  cus¬ 
tomers  in  the  system,  the  expected  time  in  the  system,  and  the  rate  of  missed  customers  —  can 
undergo  a  sudden  collapse  as  the  result  of  small  changes  in  the  arrival  rate,  the  overhead  rate,  or 
the  queue  capacity.  The  system  has  the  interesting  property  that  increasing  the  queue  capacity  n 
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can  decrease  performance.  In  addition  to  equilibrium  results,  we  consider  the  dynamic  behavior  of 
the  model.  We  show  that  the  system  tends  to  operate  in  either  of  two  quasi-stable  modes  of 
operation  —  one  with  low  queue  lengths  and  one  with  high  queue  lengths.  System  behavior  is 
characterized  by  long  periods  of  operation  in  both  modes  with  abrupt  transitions  between  them. 

We  point  out  that  the  performance  of  a  saturated  system  may  be  improved  by  dynamic  operating 
procedures  that  return  the  system  to  the  low  mode. 
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1.0  INTRODUCTION 


Batch  computer  systems  and  message  switching  systems  can  be  modeled  [1]  by 

the  M/M/l/K  queuing  system  -  customers  arrive  with  independent,  exponentially 

distributed  interarrival  times  at  an  average  rate  ^  from  an  infinite  customer 
pool;  they  wait  in  a  queue  with  finite  capacity  K;  they  are  served 
independently  by  a  single  server  with  exponentially  distributed  service  times 
at  a  constant  average  rate  ^  ;  and  they  return  to  the  customer  pool. 

Customers  arriving  at  a  full  queue  are  denied  service  and  return  immediately 
to  the  customer  pool.  As  a  first  step  in  modeling  overhead,  it  seems 
reasonable  to  modify  the  M/M/l/K  model  from  a  constant  expected  service  time 
l^i  to  an  expected  service  time  l^fc^  that  depends  on  the  the  number  of 
customers  currently  in  the  system.  In  this  paper,  we  consider  a  model  in 
which  the  expected  service  time  has  the  form  of  a  constant  plus  a  queue-length 
dependent  "overhead"  term, 

=  f(n)  +  1  ijt-,  (1) 

where  f  increases  monotonica 1 ly  with  n  and  satisfies  f(l)  =  0.  In  particular, 
we  consider  cases  in  which  f(n)  grows  linearly  and  logarithmically.  This 
model  is  an  example  of  a  class  of  models  that  has  been  studied  in  more  general 
terms  by  Courtois  and  Vantilborgh  [2]. 

Modeling  overhead  by  means  of  a  service  time  that  increases  with  the 

number  of  customers  in  the  system  could  be  appropriate  in  a  variety  of 

circumstances.  Examples  include  systems  with  scheduling  algorithms  that 

consider  attributes  of  all  waiting  customers  in  deciding  which  one  to  serve 

next  (as  opposed  to  customer  independent  algorithms  like  first-come, 

first-served),  and  systems  in  which  overhead  functions  like  compaction,  page 
Note:  Manuscript  submitted  October  5,  1979. 
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swapping,  etc.,  are  performed  more  frequently  as  the  number  of  customers  in 
the  system  increases.  The  model  can  also  be  seen  as  an  abstraction  of 
broadcast  packet  radio  systems  such  as  slotted  ALOHA  [31, (41:  tne  queue  size 
corresponds  to  the  number  of  nodes  that  have  messages  ready  to  transmit,  and 
the  service  time  corresponds  to  the  time  taken  for  a  single  successful 
transmission.  Owing  to  contention,  the  transmission  time  increases  with  the 
number  of  nodes  that  have  messages  to  transmit.  In  a  variety  of  ways,  the 
model's  dynamic  behavior  resembles  the  known  behavior  of  systems  like  ALOHA 
(3l,[4l,  and  it  may  be  that  such  systems  can  be  modeled  effectively  using  the 
simple  model  and  techniques  presented  herein. 

It  is  obvious  that  modifying  the  service  rate  from  =yt  to  (1)  will 
decrease  system  performance.  What  is  surprising,  at  least  to  me,  is  how 

non-linear  the  relationship  is:  Huge  changes  in  performance  -  as  measured  by 

mean  queue  length,  mean  waiting  time,  and  rate  of  rejected  customers  -  can 

result  from  small  changes  in  the  arrival  rate  A  »  in  the  queue  capacity  K,  or 
in  a  parameter  characterizing  the  derivative  df(n)/dn. 

2.0  THE  MODEL 

Let  p^  be  the  probability  that  n  customers  are  in  the  system  (waiting  or 
being  served).  Then  p^  is  given  [5,  p.  92 1  by 

n 

Pn  =  Po*"  TT  !/An>  ’  (2) 

m=l 

where  %  is  the  arrival  rate,  is  the  service  rate  given  that  n  customers 

are  in  the  system,  and  p^  is  the  probability  that  the  system  is  empty. 


2 


1 


(3) 


K  n 


1  + 


I  n  % 

n=l  m=l 


Now,  let  ^  be  given  by 


K 


(4) 


(1  +  *(n  -  D) 


For  example,  if  oC  *  .01*  then  (4)  states  that  the  expected  service  time  l^<n 
is  l/jt <  when  a  single  customer  is  in  the  system  and  increases  by  IX  for  each 
additional  customer  in  the  system.  Combining  ( 2 )  —  C 4 )  yields 


Pn  =  Po/TT  °  +  "  1 


)) 


(5) 


m=l 


and 


1 


(6) 


K  n 

‘♦syn  (1  +  e((m  -  D) 

n=l  ra=l 


where  p  =  .  As  a  second  case,  we  also  consider  expected  service  times  of 

the  form 

(7) 


/* n 


— £ - 

(1  +  ji  log(n)) 


where  log(n)  is  the  natural  logarithm.  In  this  case  the  queue  distribution  is 


II 

pn  “  P0  f°Tf  (1  +  flog(m)) 

m=l 


(8) 
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with 


_ 1 _ .  (9) 

K  n 

1  +  j5"1  TT  ^  +  ^  log(m) ) 

n=l  m=l 

The  motivation  for  considering  both  (4)  and  (7)  is  that  algorithms  depending 
on  attributes  of  all  customers  in  the  queue  probably  execute  in  times  that 
grow  at  least  logarithmically  and  no  more  than  linearly  with  the  queue  size. 


t 


r 


3.0  PERFORMANCE  OF  THE  MODEL 

We  characterize  the  performance  of  the  model  described  in  Section  II  in 
terms  of  the  expected  number  in  the  system  L, 

K 

l  ■  Z np"  ’ 

n=l 

the  expected  system  time  T  (waiting  time  plus  service  time  -  from  Little's 

result  [5,  p.  171), 

T  =  L/ft(l  -  pR), 

and  the  rate  of  missed  customers  M, 

M  =  %  pR  . 

We  study  the  overall  behavior  of  the  model  by  seeing  how  L,  T,  and  M  are 
affected  by  the  parameters  A  »  K>  and  U  or  /£  . 
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3 . 1  Performance  for  Linear  Service  T imes 

We  start  with  the  case  in  which  the  expected  service  time  increases 
linearly  with  the  number  of  customers  in  the  system  (Eqs.  (4)-(6)).  For  fixed 
system  capacity  K  =  50  and  fixed  load  factor  =  .5,  Fig.  1  shows  the 

expected  number  in  the  system  L  and  the  expected  system  time  T  as  functions  of 
0(  .  Performance  is  relatively  unaffected  by  increases  in  o<  until  of  reaches 

a  threshold,  at  which  point  the  performance  suddenly  collapses  -  the  system 

saturates  with  L  approaching  K  and  with  T  growing  unbounded.  Furthermore,  as 
L  approaches  K,  p  approaches  l  so  that  the  missed  customer  rate  M 
approaches  A  .  If  the  second  term  in  l^n^  =  O/^)  +  <#/^*)( n  -  1)  is 
thought  of  as  an  overhead  term,  Fig.  1  suggests  that  there  are  regions  of 
system  operation  in  which  a  small  change  in  the  overhead  "rate"  o'  will  lead  to 
a  performance  collapse.  In  its  suddenness,  the  phenomenon  is  reminiscent  of 
thrashing  [6] . 

The  sensitive  region  of  operation  can  also  be  encountered  by  changes  in 
p.  For  example,  Fig.  2  shows  L  and  T  as  a  function  of  p  for  K  =  50  and 
A  =  .04  and  also  for  of=  0.  The  <H  =  0  curves  show  the  no-overhead  M/M/l/K 
case.  The  oC  =  .04  curves  show  that  a  small  change  in  the  arrival  rate  A  can 
result  in  a  performance  collapse. 

The  performance  of  M/M/l/K  systems  can  always  be  improved  by  increasing 

the  storage  capacity  -  in  the  K-*8®  limit  (M/M/1),  L  and  T  reach  limits  and  M 

goes  to  zero.  In  sharp  contrast,  the  performance  of  the  modified  model  can 
collapse  when  K  is  increased.  Figure  3  shows  L  a  a  function  of  K  for  three 

values  of  U oL=  0  (M/M/l/K),  41=  .03,  and  =  .06  (the  load  factor  is 

p  =  .5  in  all  cases).  As  K  increases,  the  curves  for  pf  =  .03  and  <K  =  .06 
eventually  increase  as  fast  as  K.  The  effect  can  be  seen  in  different  terms 
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in  Fig.  4,  which  shows  p  vs.  K.  For  the  t'./M/l/K.  case  (<*=  0),  p 

K  K 

decreases  raonotonical ly  and  is  asymptotic  to  zero.  But  when  £X  ^  0 ,  p  (and 

K 

therefore  M)  eventually  increases  with  K,  owing  to  the  possibility  of  large 
service  times,  and  in  all  cases  p^  is  asymptotic  to  one  instead  of  zero! 

These  somewhat  surprising  results  suggest  that  there  are  situations  in  which 
adding  memory  to  a  system  will  excacerbate  rather  than  solve  a  performance 
prob lem. 

3 . 2  Performance  for  Logarithmic  Service  Times 

One  might  suspect  that  the  performance  collapse  would  be  much  less 
dramatic  if  the  increase  in  expected  service  time  were  much  less  than 

linear.  However,  similar  effects  occur  for  logarithmic  increases  (Eqs. 
(7)-(9)),  although  for  considerably  larger  values  of  the  proportionality 
constant  than  was  the  case  for  c(  .  For  example,  Fig.  5  shows  L  and  T  as 
functions  of  |?  for  system  capacity  K  =  50  and  load  factor  j?  =  .5,  and  Fig.  6 
shows  L  and  T  as  functions  of  j>  for  K  =  50  and  =  .3.  By  comparing  Figs.  2 
and  6,  one  can  see  that  performance  in  the  logarithmic  case  can  change  almost 
as  much  for  a  small  change  in  as  can  performance  in  the  linear  case. 

Finally,  Fig.  7  shows  L  as  a  function  of  K  for  three  values  of  - 0 

(M/M/l/K),  .3,  and  =  .4  (the  load  factor  is  J5  =  .5  in  all  cases). 

Although  the  curves  are  not  as  steep  as  those  in  Fig.  3,  the  effect  is  still 
dramatic . 

4.0  DYNAMIC  BEHAVIOR 

Equilibrium  results  such  as  those  in  Figs.  1-7  are  interesting  and 
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informative,  but  they  shed  little  light  on  the  dynamic  behavior  of  a  system. 
For  example,  Fig.  1  predicts  that  a  system  with  K  =  50,  p  =  .5,  and  of =  .06 
will  operate  with  an  average  queue  length  of  about  49,  but  it  gives  no 
information  about  how  fast  an  in 1 1 i a  1 ly-emp ty  system  will  deteriorate. 

Indeed,  equilibrium  results  like  Fig.  1  can  be  misleading,  since  it  is  easy  to 
get  in  the  habit  of  assuming  thaL  a  short-term  time  average  of  the  queue 
length  will  increase  at  a  steady  rate  until  the  equilibrium  expected  value  is 
reached  and  that  the  system  will  thenceforth  operate  with  queue  lengths  close 
to  the  equilibrium  expected  value.  The  performance  collapse  model  is  a 
dramatic  counterexample  to  such  thinking. 

A  first  hint  at  the  system's  dynamic  behavior  can  be  obtained  by  studying 
Fig.  3.  The  shape  of  the  curves  suggest  that  an  initially  empty  system  might 
operate  with  good  performance  for  quite  some  time  and  then  degrade  suddenly 
after  statistical  fluctuations  have  caused  the  queue  length  to  exceed  some 
threshold  value.  More  precisely,  when  the  system  is  in  a  state  with  queue 
length  n  f  0,  the  relatively  likelihood  of  a  departure  vs.  an  arrival  is  given 
by  the  ratio  j> ^  A  departure  is  more  likely  than  an  arrival  when 

1  and  vice  versa.  Since  decreases  monotonical  ly  as  n  increases 
(see  (1)),  there  will  be  a  bounded  region  n  =  fl , . . .  ,nA  in  which  ^  ^  1  and 
there  is  a  tendency  for  the  queue  length  to  decrease,  just  as  there  will  be  a 
region  n  =  -[n^ , . . . , k\  in  which  1  and  there  is  a  tendency  for  the  queue 

length  to  increase.  Thus,  in  an  initially  empty  system  there  will  be  a  kind 
of  "pressure"  that  keeps  the  queue  length  down  until  statistical  fluctuations 
eventually  cause  it  to  reach  n^ ,  at  which  point  the  pressure  reverses  and 
tends  to  drive  the  queue  length  up.  Later,  statistical  fluctuations  should 
cause  the  queue  length  to  drop  low  enough  for  the  pressure  to  reverse  again, 
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etc.  One  can  see  that  the  system  will  tend  to  operate  in  one  of  two 

quasi-stable  modes  -  a  "low  mode"  with  small  values  of  n  and  a  "high  mode" 

with  large  values  of  n.  The  threshold  between  queue  lengths  in  the  two 


modes  occurs  at  p^  =  ^ lj =  1,  namely  at 


for  the  linear  overhead  case  (see  (4)).  Thus,  as  ot  increases,  the  threshold 
decreases  monotonically  and  is  asymptotic  to  1.  For  small  values  of  Oi  or  p, 
may  satisfy  n^>K,  in  which  case  there  is  no  high  mode.  In  the 
terminology  of  [2],  n^  is  a  stochastically  unstable  congestion  of  locally 
minimum  probability  (p  has  a  local  minimum  in  the  vicinity  of  n  =  n ^ ) . 

I  confirmed  the  foregoing  by  studying  the  dynamics  of  a  variety  of  test 
cases  by  means  of  simulation.  Here  are  the  results  of  one  such  run,  conducted 
for  the  case  of  linear  overhead  with  K  =  40,  p  =  .5,  and  o(  =  .06  (Fig.  3 
predicts  an  equilibrium  average  queue  length  of  about  30):  After  starting  at 
n  =  0,  the  queue  length  didn't  exceed  n  =  9  during  the  first  4100  transitions 
(arrivals  or  departures).  The  average  queue  length  during  this  period  was 
1.52.  The  queue  length  then  went  up  and  hovered  in  the  vicinity  of  n  -  17  for 
about  320  transitions  (for  this  case  the  mode  threshold  is  n^  =  17.66), 
after  which  it  went  up  quickly,  hovered  in  the  vicinity  of  n  =  38  for  about 
the  next  14,000  transitions,  and  then  came  down  again  to  the  vicinity  of 
n  =  0.  The  average  queue  length  during  the  high  mode  was  38.15  and  the 
shortest  queue  length  was  23.  The  overall  queue  length  average  was  29.9.  In 
general,  for  fixed  values  of  K  and  p,  increasing  o(  results  in  a  greater 
portion  of  time  spent  in  the  collapsed  state. 

One  can  study  this  more  formally  in  terms  of  first  passage  times  in  the 
embedded  Markov  chain  on  the  integers  n  =  ^0 , 1 , . . . ,k| .  The  transition 
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probabilities  p.  .  from  states  n  =  i  to  states  n  =  j  are 
1  >  J 


II 

o 

CL 

1  , 

II 

,  l^j^K-1 

P3,j+I 

a 

,  1  <■  K-l 

PK,K-1 

=  i  , 

Pi,j  = 

0 

.  i  /  j+1 

(11) 


Let  m.  .  be  the  expected  number  of  transitions  from  a  time  the  system  is  in 
i.J 

state  n  =  i  until  it  first  reaches  state  n  =  j.  Since  the  embedded  Markov 
chain  is  irreducible  and  recurrent,  the  bk  ^  are  given  1 7 ,  p.  243]  by  the 
system  of  equations 


m .  . 


1  *  J 


+  Pi.k  "Vi 


(12) 


For  the  transition  probabilities  given  by  (11),  the  solution  of  (12)  is  [8] 


m.  . 


h*l 


(13) 


for  0^i<  j^K,  where  pn  =  "  * ^n '  T°  obta^n  mij  ^or  i  7  j »  one 

writes  1/p.  for  p  in  (13)  and  P  ,  and  then  computes  m,  .  .  . .  For 
JL-n  in  n  L-i  ,L-j 

example,  given  the  parameter  values  used  in  the  simulation  mentioned  above 
(K  =  40,  o  =  .5,  and  c(  =  .06),  the  expected  values  m^  ^  and  m^  ^  are 
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6354  and  15026  respectively.  The  observed  values  in  the  (single  run) 
simulation  were  about  4100  and  14000  respectively. 


Since  a  change  from  one  of  the  quasi  stable  inodes  to  the 
by  a  relatively  long  period  of  operation  in  the  new  mode,  we 
q  as  approximate  measures  of  the  relative  amounts  of  time 
low  mode  and  high  mode  respect ivel> ,  and  we  define 


oth’  -  is  f ol lowed 
tak  mn  and 
spent  in  the 


F  =  "V.O 

n 

"V.O  +  ™0,K 


(14) 


as  an  approximate  measure  of  the  fraction  of  time  spent  in  the  high  mode.  The 
mode  fraction  and  the  transition  point  n^  ( Eq .  (10))  provide 
information  about  the  model's  dynamic  behavior,  and  can  be  used  to  supplement 
equilibrium  results  such  as  those  in  Figs.  1-7.  As  an  example,  for  linear 
overhead  with  parameters  K  =  50  and  .5,  we  plot  F^  and  n^  as  functions 

of  o(  in  Fig.  8  (compare  Fig.  1).  As  pf  increases,  the  mode  threshold  decreases 
and  the  system  spends  an  increasing  fraction  of  time  in  the  high  mode. 

It  is  important  to  realize  that  the  system  can  continue  to  spend  long 
periods  in  the  low  mode  even  when  F^  is  close  to  1.  For  example,  from  Fig. 

8  we  see  that  F^  =  .99  when  0(  =.056,  but  the  expected  first  passage  time  is 
still  large  at  m^  =  22,245  transitions  (more  than  10,000  customers).  Of 
course  m^Q  q  is  about  100  times  longer,  but  the  large  value  of  m^ 
suggests  the  interesting  possibility  of  a  dynamic  operating  procedure  that 
leads  to  overall  good  performance:  Values  of  F^  close  to  one  correspond  to 
equilibrium  situations  in  which  there  is  a  large  probability  of  a  full 
system.  (For  example,  Fig.  4  shows  that  p^^  is  almost  .5  when  cC  =  .06, 

K  =  50,  and  0=  .5.)  In  such  cases,  many  customers  will  be  turned  away  at 
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rates  Pj,  for  long  periods  of  time  on  the  order  of  m^  ^ .  It  makes  more 
sense  to  turn  away  all  customers  for  the  relatively  short  time  it  would  take 
to  serve  all  customers  currently  in  the  system,  and  then  to  resume  accepting 
customers  when  the  system  is  emptv. 

5.0  DISCUSSION 

Intuition  about  the  performance  of  computer  systems  is  notoriously  bad  - 

we  always  seem  to  underestimate  the  powerful  combinatorial  effects  involved. 
Queuing  models  help  because  among  other  things  they  warn  that  seemingly 
innocent  modifications  can  have  disastrous  effects.  The  model  considered  in 

this  paper  has  this  quality  -  it  shows  how  a  small  amount  of  overhead  per 

customer  can  choke  the  system,  and  it  shows  how  more  memory  is  not  always  the 
answer  to  poor  performance. 

Furthermore,  the  model  shows  how  equilibrium  results  can  be  misleading 
with  respect  to  dynamic  behavior,  and  how  they  can  can  obscure  the  possibility 
of  performance  gains  available  by  means  of  dynamic  operating  procedures. 
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Fig.  2  —  Expected  number  of  customers  L  and  expected  system  time  T  (in  units 
of  1/A)  vs.  load  factor  p  for  linear  overhead  factors  a  =  .04  and  a  =  0  with  system 
capacity  K  =  50.  The  a  =  0  curves  show  performance  for  the  overhead-free 
M/M/l/K  case. 


Fig.  6  —  Expected  number  of  customers  L  and  expected  system  time  T  (in 
units  of  1/X)  vs.  load  factor  p  for  logarithmic  overhead  factor  0  =  .3  with 
system  capacity  K  =  50. 
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